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Abstract. The purpose of the present study 
was to examine the effect of individual and 
school average achievement on students’ 
self-concept in science. The data of the last 
four Trends in International Mathematics 
and Science Study (TIMSS) cycles were 
used, and a two-stage individual data 
meta-analysis was conducted to examine 
the relationship between achievement 

and self-concept in science. In the first 
stage, multilevel structural equation 
modeling was performed for each 

dataset. Furthermore, the effect sizes were 
synthesized via meta-analysis with a three- 
level random-effect model. In addition, 
moderation analysis was conducted for 

the year, grade, and degree of stratification 
in countries. The results suggested a 
positive and significant effect of individual 
science achievement on the self-concept of 
students in science. On the other hand, the 
relation of the average school achievement 
was negative. The results were discussed by 
considering the big-fish-little-pond effect. 
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Introduction 


Students’ self-concept is one of the significant educational constructs 
that influence their learning outcomes (Bong & Skaalvik, 2003). Self-concept 
is about individuals’ self-beliefs about their ability in an academic domain 
(Ferla et al., 2009). Students can evaluate their abilities by using different 
references such as comparing their performances to past experiences or 
their peers’ performances (Bong & Skaalvik, 2003; Wigfield & Eccles, 2000). 
The big-fish-little-pond effect (BFLPE) theory argues that social comparison 
is an important reference point for students to evaluate their ability in an 
academic domain. In other words, according to the theory, students tend to 
compare themselves with their peers like their classmates or schoolmates and 
adopt self-beliefs based on the results of this comparison (Marsh, 2004). The 
theory underlines that those external references may influence self-concept, 
and achievement is an important external reference for students. Hence, the 
achievement of the group that students are involved in becomes an impor- 
tant predictor of students’ self-beliefs. As the average group's achievement 
increases, individuals’ self-concept decreases (e.g., Marsh et al., 2008; Seaton 
et al., 2010). For instance, Chen (2022) examined the BFLPE in mathemat- 
ics for Hong Kong students with Trends in International Mathematics and 
Science Study (TIMSS) 2019 data and supported that group achievement 
negatively affects self-concept not only for eighth graders but also for fourth 
graders. Besides, even though the BFLPE was significant for both grades, it 
was stronger for eighth graders. Marsh et al. (2014) also showed that the 
BFLPE is lower for younger students in their study in which they examined 
the BFLPE across diverse cultures. In another study, Wang (2020) examined 
the BFLPE in mathematics with a cross-national analysis, using TIMSS 2015 
data for eighth and fourth graders. Multilevel structural equation modeling 
showed that the BFLPE was significant for most participant countries for 
both grades. Fang et al. (2018) conducted a meta-analysis to synthesize the 
BFLPE. They involved 56 effect sizes from 33 studies and found a significant 
and small mean effect size (8 = —0.28). 

Nowadays, researchers tend to be interested in synthesizing the 
BFLPE from complex surveys like TIMSS with a meta-analytic perspective. 
For instance, Parker et al. (2021) used TIMSS cycles from 2003 to 2015 and 
analyzed 130 BFLPE sizes for grades four and eight with a Bayesian random 
meta-regression. According to the findings, the BFLPE was negative and sig- 
nificant except in two cases. Furthermore, there was a large, positive relation 
between the BFLPE and ability stratification. In a recent study, Compos et al. 
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(2022) examined the BFLPE in mathematics by using international large assessments (e.g., TIMSS, PISA). They used 
a two-stage parameter-based meta-analysis to estimate the effect size and confirmed the significant negative ef- 
fect of group average achievement on students’ mathematics self-concepts. Moreover, the findings suggested no 
significant differences between years in terms of the BFLPE in mathematics. In addition, Guo et al. (2018) discussed 
that most studies that explored the BFLPE focus on mathematics. More recently, the BFLPE has been examined in 
different domains like reading. According to the author's knowledge, limited research examines the BFLPE in sci- 
ence with a meta-analytic perspective. Hence, this study aimed to contribute to the literature by adding results for 
science. To sum up, the present study aimed to examine the effect of individual and school average achievement 
on students’ self-concept in science by synthesizing the results from the last four cycles of TIMSS. 


School Stratification 


Educational systems can shape students’ school environments, which in turn influences their self-beliefs. To 
illustrate, Guo et al. (2018) examined the BFLPE in mathematics, science, and reading among 15 Organization for 
Economic Co-operation and Development (OECD) countries by considering school stratification influence. They 
used TIMSS and PIRLS 2011 databases, and intraclass correlation (ICC) as a measurement of the degree of school 
stratification. Their results suggested a negative and significant correlation between the BFLPE and ICC, which 
means that highly stratified systems feed the negative effect of school average achievement on the self-concept 
of students. In addition, comparing the magnitude of the correlation coefficients suggests that the effect in sci- 
ence is perceived as weaker than in mathematics. In a recent study, Parker et al. (2021) also examined how school 
stratification predicts the BFLPE of students. They used TIMSS datasets for 2003, 2007, 2011, and 2015 from OECD 
countries. They meta-analyzed 130 BFLPE and ICC estimates in mathematics and suggested a large relationship 
between ability stratification and the BFLPE. In other words, when students with similar abilities are encouraged to 
enter the same schools, the negative effect of group achievement average on students’ mathematics self-concept 
becomes higher than usual. On the other hand, students in selective schools may tend to feel the school accom- 
plished their success. This is assimilation and is termed the ‘reflected glory effect’ Namely, students may think that if 
they are good enough to be in this school, they can be successful at the given task (Marsh et al., 2000). For instance, 
Parker et al. (2010) examined the effects of ability grouping on students’ mathematics self-concepts and supported 
the idea that school average can have both a negative contrast effect (BFLPE) and a positive assimilation effect. 
Cultural characteristics are one of the factors affecting which of these school average achievement effects will 
dominate. For example, collectivistic countries tend to have a smaller contrast effect than individualistic countries 
(e.g., McFarland & Buehler, 1995). Another factor that determines whether the BFLPE would have a contrast effect 
or assimilation effect is socio-economic status (Parker et al., 2018). They discussed that when the BFLPE is large, 
students from a low socio-economic background tend to have a higher self-concept than expected. To sum up, 
school stratification is a critical issue that should be considered while investigating the BFLPE. However, there is little 
research about it (Parker et al., 2021). Consequently, in the current study, countries’ school stratification degree was 
treated as a moderator variable of the relation of self-concept to both school average and individual achievement. 


Research Focus 


This study aimed to examine the effect of individual and school average achievement on students’ self-concept 
in science and the effect of grade, year, and ability stratification on these relations by synthesizing the results from 
the last four cycles of TIMSS: 2007, 2011, 2015 and 2019 data. To reach the aim of the study, the following research 
questions were addressed: 


(1) To what extent is the level of achievement related to the self-concept of students in science? 

(2) Does school average science achievement relate to the science self-concept of students (BFLPE)? 

(3) Do year and grade moderate the relation of science self-concept to school average and individual 
achievement of students? 

(4) Does the degree of school stratification moderate the relation of science self-concept to school average 
and individual achievement of students? 
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Research Methodology 
Data 


TIMSS is an international exam which is carried out by the International Association for the Evaluation of 
Educational Achievement (IEA). It is administered every four years according to a schedule and focuses on fourth 
and eighth graders’ mathematics and science achievement. Additionally, it also provides data on non-achievement 
variables such as socioeconomic status, demographic characteristics, and home and class environment (Interna- 
tional Association for the Evaluation of Educational Achievement, n.d.). 

The data of TIMSS's last four cycles for both eighth-grade and fourth-grade students were used in the cur- 
rent study. For eighth-grade measures, some countries teach science in separate areas like biology, chemistry, or 
physics. Hence, they were not included in the study. Only the general science data of the survey is included in the 
analysis. Since there were no exclusion criteria except for measuring the self-concept of science in general form, 
all the available data, 345 data sets from 2007 to 2019, were used. There were 1,053,438 fourth-grade students and 
795,936 eighth-grade students. In Figure 1, The PRISMA diagram (Moher et al., 2009) presents the included and 
excluded data sets of the current study. Table 1 summarizes the number of countries for each year. 


Table 1. 
Number of Countries For Each Year 


Number of countries 
Year 
Fourth grade Eighth grade 

2007 40 34 

2011 57 34 

2015 50 32 

2019 63 30 
Figure 1 
The PRISMA diagram 


Records identified from*: 
Identifying complex survey 
(TIMSS) 

(n =395) 


= Data excluded: 
Reports assessed for eligibility Science measured separately 


(n = 395) (n =48) 
CFA did not confirm the 
structure (n = 2) 


Data of included in analysis 
(n = 345) 
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Instrument and Procedures 
Science self-concept 


Three items; (i) “I usually do well in science’ (ii) Science is not one of my strengths, (iii) | learn things quickly 
in science were handled as students’ science self-concept (Wang, 2015). The confirmatory factor analysis was 
conducted for each country to examine the validity of the measurement. All the data confirmed the structure of 
the self-concept (CFI >.90, RMSEA <.08, SMR <.08) except for two datasets. Hence, the data from the exception 
countries were removed from the analyses. 


Science achievement 


TIMSS assesses students’ science achievement in four domains: biology, chemistry, physics, and earth sci- 
ence, and three cognitive areas: knowing, applying, and reasoning. Besides, achievement scores are treated as 
five plausible variables in databases. 


School stratification degree 


To assess countries’ degree of school stratification, the ICC of science achievement was calculated for each 
country. ICC shows us how the total variance in science achievement is explained by school differences. If ICC 
approaches one, it means a high school stratification system; students may be rank-ordered according to their 
achievement scores and are entered into the school. In contrast, ICC close to zero means that students are randomly 
assigned to the schools (Parker et al., 2021). 


Data Analysis 


Two-stage individual data meta-analysis was conducted to examine the relationship between achievement 
and self-concept in science. Campos et al. (2023) suggested that meta-analysis of the data with complex sampling 
designs can be complex, and statistical problems can occur, such as handling survey weights or clustered data 
structure while conducting one stage of individual participant meta-analysis. Moreover, they suggested a two-stage 
individual participant data analysis for complex survey designs. According to their proposal, the first stage comprises 
individual data analysis from the studies and the second stage makes a meta-analytic synthesis of effect sizes. 

In the current study, multilevel structural equation modeling analysis was performed for each of the data sets 
for the first stage of meta-analysis. Since achievement was assessed via five plausible variables, each analysis was 
run five times and combined for the data estimates. The quadratic component of science achievement was also 
included in the model at Level 1. At Level 2, the school average of science achievement was obtained by rounding 
the grand mean. Equation 1 and 2 represents a summary of the model. 


scwij = Bwithinachwij+ B2(achij)? + «ij (1) 
scbj = Bbetweenachbj+ 6j (2) 
swij: within level self-concept, sbj: between level self-concept, achij: student science achievement in group j, 
achbj: the average science achievement for the group. 


The R programming language and Mplus software were used for data analysis. The packages of EdSurvey, 
MplusAutomation, tidyverse, and rlang were installed in R. The codes were adopted from Wang (2020). The stan- 
dardized effect sizes were calculated by using equations 3 and 4 (Wang & Bergin, 2017). 


ES=2 x B x SD(science)/SD(scij) 3 
BFLPE ES=2x (Bbetween- Bwithin) x SD(science)/SD(scij) 
6: unstandardized regression coefficient, SD: standard deviation 


S 
peor 
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In Step 2, a meta-analysis was performed for (1) science achievement within the level, and (2) the BFLPE sepa- 
rately. Three-level random-effects models with the sampling variance for each effect size (Level 1), variation effect 
sizes within countries (Level 2), and between countries (Level 3). The analysis was conducted using the metafor 
package in R. The codes were adopted from Campos et al. (2023). 


Research Results 
Effect Size 


Before calculating the pooled effect size, Cochran’s Q was used to examine the heterogeneity. It is the classic 
method for assessing heterogeneity in meta-analyses. Using a chi-squared distribution, it calculates the prob- 
ability that, if significant, indicates greater diversity between studies than within subjects within a single study 
(Higgins & Green, 2008). The results of Cochran’s Q statistics suggested heterogeneity among studies for the BFLPE 
(Q [343] = 2195.58, p <.001) and the effect of within-level achievement (Q[343] = 9202.89, p <.001). According to 
the I? values of the variables, the magnitude of the heterogeneity was 93.51% for the effect of science achievement 
and 85.56% for the BFLPE. Hence the heterogeneity is high for both variables (Borenstein et al., 2009). The three- 
level random-effect model suggested a significant negative BFLPE (7 =-.27) and a significant positive achievement 
effect (ji = .60). The findings are presented in Table 2. 


Table 2 
Average Effect Sizes and Heterogeneity 


ii (se) 95%Cl a? o? P... P. P a QE (af) 
BFLPE -27(01) — =.29, -.25 19 04 82.57 299 85.56 2195.58 (353) 
Within-level linear effect 60(.01) 58, 63 a 03 93.51 234 95.56 9202.89 (353) 


i= Weighted average effect siz, 0”... = Variation between effect sizes within the countries, 0? =Between-country variation,  ,., ? ., and are the cor- 
responding heterogeneity indices, QE= test fot heterogenity 


Analysis of Moderator Variables 


After calculating the general effect size, the grade was added to the model as a moderator of the variables. 
Q statistics showed that the moderator is significant for both the BFLPE (Q [1] = 38.86, p < .01) and within-level 
achievement (Q [1] = 73.36, p < .01). In other words, there was a significant difference between fourth and eighth 
graders in terms of the effect of not only individual achievement but also school average achievement on science 
self-concept. According to the results, the relationship between achievement and self-concept of eighth-grade 
students is larger than the relationship between achievement and self-concept of fourth-grade students. In an- 
other model, the year was considered as a moderator of the outcome variables. According to the findings, the year 
was not a significant predictor of the BFLPE. It was significant for science achievement. The highest score was for 
2007 whereas the lowest score was for 2019. Finally, the ICC of science achievement was added to the model as 
a moderator to examine the stratification effect. According to the results, ICC was only significant for within-level 
achievement. The findings of each moderator analysis are presented in Table 3. 


Table 3 
The Findings of Moderator Analyses 


Moderator Icc Year Grade 
BFLPE Within level BFLPE Within level BFLPE Within-level 
linear effect linear effect linear effect 
fi (se) -.24 (.03)* 54 (.02)* -.29 (.02)* 70 (.02)* -.22 (.01)* .53 (.01)* 
95 % Cl -.29, -.19 A8, .59 -.34, -.24 .65, .75 -.25, -.18 50, .56 
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Moderator ICC Year Grade 

BFLPE ——fneareftect. «= «BFLPE nearest BFEPE nerf 
ICC -.03 .07* - - - - 
2011 [95 % Cl] : - -.03 [-.01, .03] -.10 [-.16, -.04]* : - 
2015 [95 % Cl] . - 04 [-.03, .10] -.13 [-.20, -.07}* - : 
2019 [95 % Cl] - : 05 [-.01, .11] -.16 [-.22, -.09]* : - 
Grade 8 [95 % Cl] - - -14[-18,-.09]* —.19.[.14, 23)" 
o?.. 19 21 18 20 18 19 
o*, 04 .04 04 .04 .04 03 
Res 82.47 93.09 80.92 92.31 79.80 92.45 
. 3.07 2.68 3.41 2.92 3.35 2.44 
Psa 84.54 95.77 84.33 95.24 83.16 94.89 
QE (df) 2180.92 (352)* 8864.79 (352)** ——- 2053.54 (352)* 8478.97 (352)* 1970.32 (352)* 6655.12 (352)* 
QM (df) 1.32 (1) 6.05 (1)* 3.49 (3) 27.67 (3)* 38. 86 (1)* 73.36 (1)* 


fi = Weighted average effect siz, 07,” = Variation between effect sizes within the countries, 0,2 = Between-country variation, /,.”, 1’, and are 
the corresponding heterogeneity indices, QE= test fot heterogenity, QM = Test of Moderator, “p <.07 


Discussion 


The current study aimed to synthesize the effect of individual and group average achievement on the self- 
concept of students in science. To achieve this aim, the last four cycles of TIMSS data for grade four and grade eight 
were used. Concerning Research Question 1 (To what extent is the level of achievement related to the self-concept 
of middle school students in science?), results suggested a significant and positive linear effect on students’ science 
self-concept within the level. The magnitude of the pooled effect size was 0.54 and the heterogeneity of the effect 
size was extremely high within countries (/? = 93.09%). Contrariwise, there was little heterogeneity between countries 
(P = 2.68 %). Research Question 2 was related to the BFLPE. Results suggested that school average achievement 
has a significant and negative effect on students’ science self-concept. In other words, the findings supported the 
BFLPE theory for science. According to three-level random-effects models, although most of the heterogeneity 
lies within countries ( = 82.47%), heterogeneity also exists between countries (/? = 3.07%). In respect thereof, it 
will be useful to mention the findings of the first stage. According to the multilevel structural equation modeling 
analyses, the achievement was positively significant for science self-concept for nearly all countries for each grade 
within the level, and the school achievement average (BFLPE) was negatively related to the science self-concept of 
students for most of the countries. This finding is parallel to previous studies’ results. For instance, Guo et al. (2022) 
examined the BFLPE in three different domains, mathematics, science and reading, across 15 countries and found 
that the between-country variance is too small. Moreover, they explored the relationship between the degree of 
ability stratification and the BFLPE at the country level and suggested that the relationship for mathematics is 
stronger than the relationship for science. Supporting this idea, in a recent study, Campos et al. (2023) synthesize 
the size of the BFLPE in mathematics with three-level random-effects models and suggest that most of the het- 
erogeneity lies between countries (/? = 78%). Hence in a further study, the BFLPE can be examined with individual 
data meta-analysis for both science and mathematics over the same data sets. 

In addition, moderator analyses were conducted to understand the role of year and grade on the heterogene- 
ity of the effect of both individual and school average achievements on students’ self-concept. Findings supported 
non-significant heterogeneity among cycles of TIMSS regarding the BFLPE. Namely, the year did not contribute 
significantly to the heterogeneity of the school average effect size. On the other hand, it was a significant modera- 
tor of the linear effect of science achievement. While the lowest effect size was for 2019, the highest one was for 
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2007. Furthermore, the grade was a significant predictor for the BFLPE and within-level achievement; eighth-grade 
students tend to feel the effects more strongly than fourth-grade students. This was an expected finding because 
previous studies also offer a smaller BFLPE for younger students (e.g., Chen, 2022; Marsh, 2014). 

Another aim of the current study was to examine whether the degree of school stratification controls the 
relationship between achievement and self-concept in science. Therefore, the ICC of science achievement for each 
country was considered as a moderator for the effect sizes both for school average achievement (BFLPE) and indi- 
vidual achievement. According to the results, there was no significant moderation impact on the BFLPE. This was a 
surprising finding since it is expected that selective educational systems tend to cause a negative BFLPE on students’ 
self-concept. In addition, previous results suggested negative relations between the degree of stratification and the 
BFLPE. For instance, in their study, Parker et al. (2021) meta-analyzed 28 OECD countries’ data in mathematics and 
suggested a large and negative significant relation between ICC and the BFLPE. In another study, Guo et al. (2018) 
examined this relation not only for mathematics but also for science and reading and confirmed the significant and 
negative impact of school stratification on the BFLPE of students. They also demonstrated that the negative effect 
is larger in mathematics compared to science. However, they examined this effect only in OECD countries. These 
countries mostly show individualistic characteristics, and individualistic countries tend to have a larger negative 
effect of school average achievement on the self-concept of students than collectivistic cultures (e.g., McFarland 
& Buehler, 1995). On the other hand, the data of the current study involves not only OECD countries, but all TIMSS 
participant countries involved in the analyses, so in some countries, a positive assimilation effect of the BFLPE 
can occur. For instance, in the reflected glory effect, the student can perceive the group success as a reference to 
evaluate their ability in an academic domain (Marsh et al., 2000). Therefore, these cultural or economical differences 
may cause a drop in the significance of school stratification. In a further study, the school stratification effect can 
be examined by considering the cultural differences among countries. 

To sum up, the effects of both individual and school average achievement were meta-analyzed in this study. 
The last four cycles of TIMSS data were searched, and 355 data were involved in the analysis. This study makes 
several contributions. Firstly, considering the relationship between self-concept and other educational outcomes 
(e.g., Sudrez-Alvarez et al., 2014), it is important to examine the factors that influence students’ self-concepts, of 
which achievement is a prominent predictor. Hence, this study examined the effects of achievement in two forms: 
the linear individual effect within the level, and the school average form (BFLPE). Next, although the BFLPE is mainly 
explored for mathematics in the relevant literature (Guo et al., 2018), the magnitude of the effects can vary with 
the academic domain. To the author’s knowledge, this is the first study that synthesizes the BFLPE with individual 
participant data meta-analysis in science. 


Conclusions and Implications 


The study presented an insightful examination into the influence of individual and school-average achievement 
on students’ self-concept in science, using data from multiple cycles of TIMSS. The results illuminated the nuanced 
relationship between academic achievement and self-concept, particularly emphasizing the significant impact of 
both individual science achievement and school-average achievement on students’ perceptions of themselves in 
the scientific domain. The results concerning the BFLPE in science which are consistent with previous research in 
mathematics and provide a comparative perspective across academic domains. The study’s examination of the 
BFLPE in science education, an area with limited prior meta-analytic investigation, adds depth to the literature. 
Recognizing the impact of group achievement on individual self-beliefs could lead to tailored interventions or 
teaching methods to counteract negative effects. 

In addition, the study examined the moderating roles of grade, year, and school stratification, providing insights 
into how these factors influence the associations between achievement and self-concept. The study highlights 
the influence of grade level on the relationship between achievement and self-concept. Understanding that older 
students may be more affected by group performance could lead to age-specific interventions aimed at increasing 
self-confidence, especially in subjects where the BFLPE is more apparent. 
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